System for text assisted telephony

ABSTRACT

An arrangement for providing captioned telephone service is provided which permits persons who are hard of hearing to receive captioning of their telephone communications as they need it. A personal interpreter/captioned telephone device can dial a relay on a second telephone line while the assisted user if conversing with a hearing user over a first telephone line. The second telephone line connects to a relay which provides voice to text translation service and returns a text stream to the captioned telephone device.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation-in-part of application Ser.No. 09/783,679, filed Feb. 14, 2001, pending, which is acontinuation-in-part of application Ser. No. 09/288,420, filed Apr. 4,1999, now U.S. Pat. No. 6,233,314, which is a continuation ofapplication Ser. No. 08/925,558 filed Sep. 8, 1997, now U.S. Pat. No.5,909,482.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0002] Not applicable.

BACKGROUND OF THE INVENTION

[0003] The present invention relates to the general field of telephonecommunications. In more particular, the invention relates to systems toassist telephone communications by those persons who are deaf, hard ofhearing, or otherwise have impaired hearing capability.

[0004] Most modern human communications in both social and businessenvironments takes place through sound communications. Yet within modernsociety there are many persons who have attenuated hearing capability.To assist those persons in making use of our telephonic communicationsystem built for the hearing majority, there has been developed a systemof telephone communication which has been principally used by the deafcommunity. That system makes use of a category of device known variouslyas a telecommunication device for the deaf (TDD), text telephone (TT) orteletype (TTY). Current TDDs are electronic devices consisting of a keyboard and a display as well as a specific type of modem, to acousticallyor directly couple to the telephone line. Modern TDDs permit the user totype characters into their keyboard, with the character strings thenencoded and transmitted over the telephone line to be displayed on thedisplay of a communicating or remote TDD device.

[0005] Most TDD communication is conducted in an idiosyncratic codespecific to the community of TDD users. This code, known as Baudot,evolved historically at a time when many telecommunication devices forthe deaf were based on mechanical or electromechanical devices ratherthan the current technology based on digital electronic components.Accordingly, the Baudot protocol was constructed for a set ofconstraints which are no longer relevant to present date devices. Theoriginal Baudot protocol was a unidirectional or simplex system ofcommunication conducted at 45.5 Baud. The conventional Baudot characterset was a character set consisting of 5 bit characters and the systemencodes the bits of those characters in a two-tonal system based oncarrier tones of 1400 and 1800 Hertz.

[0006] The system of TDD communications is widely used and in fact hasbecome indispensable to the deaf community throughout the industrializedworld. Deaf persons extensively communicate with their neighbors andwith other deaf and hearing people remotely, using the TDD system. Inaddition, systems have been developed to facilitate the exchange ofcommunication between the deaf community and hearing users who do nothave access to or utilize a TDD device. In the United States, telephonecompanies have set up a service referred to as a “relay.” A relay, asthe term is used herein, refers to a system of voice to TDDcommunication in which an operator, referred to as a “call assistant,”serves as a human intermediary between a hearing user and a deaf person.Normally the call assistant wears a headset that communicates by voicewith the hearing user and also has access to a TDD device which cancommunicate to the deaf user using a TDD appropriate protocol. In normalrelay operations in the prior art, the call assistant types at a TDDkeyboard the words which are voiced to her by the hearing user and thenvoices to the hearing user the words that the call assistant sees uponthe display of his or her TDD. The call assistant serves, in essence, asan interpreting intermediary between the deaf person and the hearingperson to translate from voice to digital electronic forms ofcommunication.

[0007] A system to assist users of the telephone system who are hard ofhearing but not deaf has been described. This system, sometimes referredto as text enhanced telephony, makes use of the existence of relays tosupplement telephone communications for users who can hear, but haveattenuated hearing capabilities. This systems includes, in its simplestembodiment, a visually readable display connected in series between thetelephone used by the assisted user and the connection to the telephonenetwork. The text enhanced telephone call is connected through a relaywhich transmits both the voice of the hearing user at the other end anda text stream of the words spoken by that user on the same telephoneline. The details of the concept and of some embodiments of that systemare disclosed in U.S. Pat. No. 6,075,842, the disclosure of which ishereby incorporated by reference.

BRIEF SUMMARY OF THE INVENTION

[0008] The present invention is summarized in that a relay system tofacilitate the translation of information and communication between deafand hearing persons includes a call assistant who re-voices the words ofthe hearing person which are spoken to the call assistant. The wordsspoken by the call assistant are recognized by a speech recognitioncomputer program which has been trained to the voice pattern of the callassistant, such that the words are promptly translated into a high speeddigital communication protocol. That high speed digital communicationmessage is then transmitted electronically promptly by telephone to avisual display accessible to the deaf person.

[0009] It is an advantage of the invention described herein that thecall assistant does not have to type most, if any, of the words spokenby the hearing person in the communication session so that the overallspeed of communications from the hearing person to the deaf person isdramatically increased.

[0010] It is an object of the present invention that the design andutilization of a relay operated in accordance with the protocolsdescribed herein permits the introduction of small hand-held personalinterpreter which will enable on the spot communications between deafpersons and hearing persons wherever the deaf persons might go.

[0011] Other objects, advantages and features of the present inventionwill become apparent from the following specification when taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0012]FIG. 1 is a schematic block diagram of a re-voicing relay.

[0013]FIG. 2 is an exterior view of a personal interpreter enabled bythe relay of FIG. 1.

[0014]FIG. 3 is a schematic block diagram of the personal interpreter ofFIG. 2.

[0015]FIG. 4 illustrates the operation of a captioned telephone servicesupported by a relay.

[0016]FIG. 5 is a schematic illustration of the concept of the two wirecaptioned telephone device connection with sound inter-connection.

[0017]FIG. 6 is a schematic illustration of the connection of a personalinterpreter/captioned telephone device connected to two telephone lines.

DETAILED DESCRIPTION OF THE INVENTION

[0018] The present invention is intended to provide an alternativearrangement for the delivery of text assisted telephone services, alsocalled captioned telephone, to a telephone system user with diminishedhearing. The present system is intended to take advantage of twodevelopments in the field of assisted telephone services, the personalinterpreter and the re-voicing relay. The arrangement for using thesecapabilities here is designed not to provide text services for theprofoundly deaf, but to provide a text assistance service for those whocan hear, but who may have diminished hearing. This service is designedto supplement rather than replace the transmission of the spoken voiceword to the assisted user. To assist understanding what is contemplatedhere, the basic technologies of the personal interpreter and there-voicing relay will be described here first, after which thedescription will return to the main topic, text assisted telephoneservices or, as they are also known, captioned telephone services.

[0019] The personal interpreter is intended to be a small portabledevice capable of delivering, with the support of a relay, text to adeaf user of any human conversation occurring in the presence of thepersonal interpreter. The personal interpreter is set up to connecttelephonically to a relay, transmit spoken words to the relay, and thendisplay for the assisted user the text of the words transmitted over thetelephone connection to the relay. To make the personal interpreter workmore seamlessly to produce more conversation-like communication, afaster relay methodology was needed. To fill this need, the re-voicingrelay was designed.

[0020] The re-voicing relay is based upon the underlying technology ofusing voice recognition software, operated by a call assistant (a “CA”),to assist in the voice to text translation inherent in providing relayservices. The re-voicing strategy is described in U.S. Pat. No.5,909,482, the disclosure of which is hereby incorporated by reference.The re-voicing relay has its first implementation in providing voice totext transcription services for the deaf. However, the relay voice totext transcription service has use for users other than those who aredeaf. It is envisioned that there are a number of hearing or partiallyhearing users who would have reason to benefit from voice to texttranscription services. Relay voice to text service might also be usefulfor any application in which it is desired to supplement voicecommunications by a text transcription of the voice spoken on thetelephone. Thus, while the operation of the relay will sometimes bedescribed here by referring to an “assisted user,” who may be deaf orhard of hearing, but who also may be a normally hearing person whosimply wants text assistance for some reason. The user at the other endof the line will be referred to here as the hearing user, simply for thepurpose of having something to call that person, even though both usersmay be hearing.

[0021] Personal Interpreter.

[0022] Shown in FIG. 2 is an illustration of what a personal interpreter10 can look like. This would be a small hand held device typically thesize of a small hardbound book. It would have a keyboard of minimalsize, but useable by a deaf person who can type. It would have a two orfour line display, but the display could be any size that convenientlyfits in the case of the device. The device would also have a key orswitch which would initiate its operation.

[0023] Shown in FIG. 2 is a schematic block diagram of the internalmechanics of the personal interpreter. The personal interpreter keyboardshown at 12 and its display as shown at 14. Inside the interpreteritself is a microprocessor shown at 16. Not shown, but included withinthe personal interpreter, would be the appropriate memory and interfacedevices so as to allow the microprocessor to be programmed and tooperate the personal interpreter and perform its functions, in a mannerwell known in the art. Also inside of the personal interpreter is amodem 18. The modem 18 is preferably a modem specifically designed forinterface with the deaf telecommunications system. Mosttelecommunications with the deaf community are conducted using a Baudottype code. One useful alternative is for the modem be designed to usethe enhanced form of Baudot communication known as “Turbo Code”(Ultratec), which is generally described in U.S. Pat. No. 5,432,837,U.S. Pat. Nos. 5,517,548, and 5,327,479, the disclosure of which ishereby incorporated by reference. Another alternative is that the modemuse a new variant of Turbo Code, one which uses higher carrierfrequencies (in the range of 3000-3500 hertz) and a faster baud rate(over 100 baud). The most preferred alternative is for the modem to usea digital communication protocol which can both transmit and receivedigital packets which contain either or both of voice and text. Theoutput of the modem is preferably wired to a cellular telephone 20included within the case of the personal interpreter 10. The cellulartelephone 20 has a suitable antenna provided on it so that it may dial acellular telephone network by radio frequency communications of the typenormally conducted by cellular telephones. The personal interpreter alsoincludes jack 28 to connect to a conventional wired or land-linetelephone line as well. The personal interpreter also includes amicrophone 22 and a speaker 24. A filter 26 connects the speaker 24 andthe microphone 22 to the telephone 20.

[0024] A brief description of the operation and functionality of thepersonal interpreter reveals the dramatic improvement and convenienceand portability that this device gives to text assisted people. Aassisted user could go into an establishment, be it a government officeor retail facility, in which there are only hearing persons. Theassisted person would carry with him or her the personal interpreter 10.The assisted user would then place the personal interpreter 10 upon acounter or other surface, open it up, and press the initiation key orstart button. The microprocessor 16 and modem 18 of the personalinterpreter then power up and act in many ways like a normal TDD deviceoperating in telecommunication standard, such as Turbo code. However,there is one critical difference. The start or initiation key furthercauses the microprocessor 16 of the personal interpreter to dial a relayto set up a relay communication session and includes in itscommunication with the relay a message, using the enhanced commandfeatures available in advanced telecommunication protocols, such asTurbo Code, to initiate a special format of relay call adapted for thepersonal interpreter. Other codes which permit command functions, suchas ASCII or CC ITT, could also be used. The first operation is toactivate the cellular telephone and direct the cellular telephone todial the number of a relay operating in accordance with the method ofthe present invention. The cellular telephone dials the relay.Obviously, no wired connection is required to allow the cellulartelephone function to establish a telephone connection with the remoterelay, but alternatively the jack 28 to a conventional telephone linecould be used. In addition, when the relay answers the telephoneconnection, the microprocessor 168 of the personal interpreter 10 isinstructed to provide command codes to the remote relay. These commandcodes, a feature possible through the use of Turbo Code, permits thepersonal interpreter to tell the relay that this is a personalinterpreter-type relay communication session. All of this can happen inthe time necessary to initiate the cellular call, perhaps two to tenseconds.

[0025] Then, the assisted person can use the personal interpreter totranslate words spoken by hearing persons in the presence of thepersonal interpreter into visually readable text. This is done by thepersonal interpreter 10 through an unseen relay. Words spoken by thehearing persons in the presence of the personal interpreter 10 arepicked up by the microphone 22. Those words are then transmitted throughthe cellular telephone 20 to the remote relay. The relay, operating aswill be described below, then immediately transmits back, in enhancedTurbo Code, a digital communication stream translating the words thatwere just spoken. The words are received by the modem 18, and themicroprocessor 16 in the personal interpreter 10, and it is displayedpromptly upon the display screen 14. If the assisted person can speak,he or she may then answer the hearing person with a spoken voice, or,the assisted person may alternatively type upon the keyboard 12. If theassisted user types on the keyboard 12, the personal interpretertransmits the communication by digital communication to the relay. Thecall assistant at the relay then reads and speaks the words typed by theassisted user which are transmitted to the speaker 22 contained in thepersonal interpreter into a voice communication which can be understoodby the hearing users. The filter 26 filters out the digitalcommunication frequencies from the sound generated by the speaker 22.Thus, in essence, the assisted person has a personal interpreteravailable to him or her at all times of the day or night wherever theassisted person is within the range of the cellular telephone system.Also, because the relay is preferably operating in accordance with thefast translation methodology described below, a very conversation-likefeel can occur in the communication session between the assisted userand the hearing persons in the presence of the personal interpreter 10.In order for this communication session to be satisfactory to thehearing users as well as the assisted person, however, the relay mustoperate exceedingly rapidly. It is, in part, to meet the need for theexceeding rapidity of this conversational style of communication thatthe re-voicing relay protocol has been designed.

[0026] Re-voicing Relay.

[0027] Shown in FIG. 1 is a relay intended to provide the voice to textcapability to support the personal interpreter, and which can alsosupport other voice to text services to provide services for assistedusers. FIG. 1 is intended to show, in schematic fashion, how such arelay system can be set up. Shown at 32 is a telephone of a hearingperson. Instead of a telephone of a hearing person, the input could alsobe the microphone of the personal interpreter translator 10 shown inFIGS. 2 and 3. The telephone of the hearing person 32 is connectedthrough a telephone line 34 to a voice input buffer 36 at the relay. Thetelephone line 34 can be an actual physical land line, or two pairbetween the telephones, or can be a cellular or other over-the-airtelephone linkage. The voice input buffer 36 is a simple buffer toensure temporary capture of the voice in the event that the callassistant gets behind and needs to buffer or delay the voice of thehearing person. In any event, the output of the input voice buffer 36 isprovided to a headset 40 where earphones 38 produce the sound of theremote speaking person in the ear of the call assistant. The callassistant is wearing the headset 40 and sitting at a computer 42 capableof communicating in an enhanced Baudot communication, such as Turbo Codeor whatever other code protocol is being used. However, typically thecall assistant does not type the words which the call assistant hears inhis or her earphone 38. Instead, the call assistant then speaks thewords which he or she hears in the earphones 38 into a microphone 39 inthe headset 40. The microphone 39 on the headset 40 is connected totransmit the voice of the call assistant to the computer 42 at which thecall assistant sits.

[0028] The computer 42 has been provided with a voice recognitionsoftware package which can recognize the spoken voice of the callassistant and immediately translate words spoken in that voice into adigital text communication stream. It is a limitation of currentlyavailable speech recognition software that the software must be trainedor adapted to a particular user, before it can accurately transcribewhat words the user speaks. Accordingly, it is envisioned here that thecall assistant operates at a computer terminal which contains a copy ofa voice recognition software package which is specifically trained tothe voice of that particular call assistant. It is also important thatthe voice recognition system be capable of transcribing the words of thevoice of the call assistant at the speed of a normal humancommunication. It has been found that a recently available commercialvoice recognition package from Dragon Systems, known as “NaturallySpeaking,” is a voice recognition software which will accomplish thisobjective and which will translate to digital text spoken words of auser at the normal speeds of human communication in conversation whenoperating on conventional modern personal computers. A voice recognitionsoftware system known as “Via Voice” from IBM provides similarfunctionality.

[0029] The computer terminal 42 of the call assistant then translatesthe text created by the voice recognition software to a modem 46 outthrough a telephone line 48 back to the display 50 located adjacent tothe assisted person. The display 50 can be a conventional TDD located atthe home of the remote assisted user, or can be the display 14 of thepersonal interpreter 10, or can be any other display or text capturedevice used by an assisted user.

[0030] For reasons that will become apparent, there is also a connectionfrom the microphone 39 of the headset 40 of the call assistant to theincoming telephone line 34 through a switch 52. The switch 52 canphysically be an electrical switch located between the microphone 39 andthe telephone lines 34 and the computer 42 or, as an alternative, it canbe a software switch operating in the computer 42 which passes the voiceof the call assistant through to the telephone lines as voice, or not,under conditions which are selected by the call assistant, by choices heor she makes at the keyboard 44 of the computer 42. The switch 52 isfunctionally a single pole double throw switch although, of course, ifthis function is performed by the computer it will be a logical not aphysical switch. In the simplest embodiment, the switch 52 is a simplesingle pole dual throw foot switch readily accessible to the callassistant which passes the voice of the call assistant from themicrophone either out onto the telephone line 34 or to the computer 42.

[0031] It is a further enhancement to the operation of the relayconstructed in accordance with the present invention that the earphones38 have noise attenuating capability. Noise canceling earphones arecommercially available today or, for this purpose, the computer 42 canbe provided with noise canceling sound generation software which wouldcreate sound transmitted to the earphone 38 so as to cancel the soundsof the call assistant's own voice. The noise attenuation or cancellationavoids distracting the call assistant, since he or she would then beless distracted by the words that he or she has spoken, and thus wouldbe less likely to be distracted from the concentration of the task ofre-voicing the sounds of the voice heard in the call assistant's ear.

[0032] Similarly, another option which would be advantageous is that thesoftware providing for the creation of the digital text string by voicerecognition be buffered in its output flow to the modem 46. Before thecomputer 42 would pass the data on to the modem 46, the data would firstbe displayed on the computer screen of the computer 42 for review by thecall assistant. The purpose of this option would be to permit the callassistant to use the keyboard to type or correct hard-to-spell words, orto create corrections of any misinterpretations created by the voicerecognition software, from the words spoken by the call assistant. It isanticipated that if such an option is utilized, it would require fairlyinfrequent use of the keyboard by the call assistant, since frequent usewould clearly slow down the through-put of the communications.

[0033] The relay of FIG. 1 can operate with normal TDDs or with apersonal interpreter as shown in FIGS. 2 and 3. In either event, thehearing person speaks in the telephone 32 and the words are transmittedthrough the telephone line 34 to the voice buffer 36. The voice buffer36, again operating under the control of the call assistant, wouldbuffer the voice signals from the hearing user as needed for the callassistant to keep up. The call assistant would hear the voice of thehearing user through the earpiece 38 and then would re-voice those samewords into the microphone 39. The words that the user speaks into themicrophone 39 would be fed to the computer 42 where the voicerecognition software, trained to the voice of the call assistant, wouldtranslate those words into a digital text stream. The digital textstream would be turned into a digital communication stream by the modem46 and passed on the telephone line 48 to a display 50 which can beobserved by the assisted user. Experience has shown that using currentlyavailable technology the delay between the time the hearing user speaksinto the telephone 32 and the time the words appear on the display 50 ofthe assisted user is a modest number of seconds.

[0034] In the reverse, when a non-speaking assisted user types onto hisor her telecommunication device, the digital signals are transmitted tothe computer 42 which displays them for the call assistant who thenvoices those words into the microphone 39 which words are thentransmitted onto the telephone line 34. Note that the presence of theswitch 52 is therefore important in this mode. Since the voice of thecall assistant serves two different functions in the operation of thissystem, the signal on the call assistant's voice must be switched sothat the hearing user 32 only hears the voice for the communicationswhich are intended to be directed to that person. The switch 52 allowsfor the voice of the call assistant only to be directed to the hearingperson at the appropriate times.

[0035] Note that if the relay of FIG. 1 is used to facilitate atranslation based on a personal interpreter such as that shown in FIGS.2 and 3, there will be only one telephone line between the personalinterpreter and the call assistant. In essence, in a modification ofFIG. 1, the telephone 32 and the display 50 would both be within thepersonal interpreter 10. There would be only one telephone line, acellular link, between the personal interpreter 10 and the callassistant. Note therefore that the voice of the call assistant and thedigital communications created by the computer 42 would then travel onthat same telephone linkage to and from the personal interpreter 10. Itis therefore important for this embodiment that the personal interpreter10 have appropriate filtering (i.e. the filter 26) to filter out thedigital communication carrier frequencies of the digital communicationsprotocol, so that they are not heard by hearing listeners in thepresence of the personal interpreter 10. The telephone line must stillcarry voice signals, however, so that the spoken words articulated bythe call assistant in response to digital instructions from the deafuser can be properly broadcast by the speaker contained within thepersonal interpreter.

[0036] The provision for filtering of the digital frequencies can bedone in any number of ways with two being the principal preferredmethodologies. If Turbo Baudot communications are conducted at theconventional Baudot frequencies of 1400 and 1800 Hertz, the personalinterpreter 10 could be provided with notch filters 26 to filter outsignals at those particular frequencies. It has been found that suchnotch filters still permit the transmission of audible andunderstandable human speech, even if they filter at those particularfrequencies. As an alternative, it is possible to change the Baudotfrequencies to those which are much higher, such as frequencies of 3000to 3500 Hertz. If this alternative is selected, the personal interpreter10 is then provided with a low pass filter which permits low frequencysounds to go to the speaker to be broadcast into the environment of thepersonal interpreter, while high frequencies are excluded.

[0037] It is also specifically envisioned that the filter of textcommunications signals from the voice signal can be done digitally orlogically rather than by analog filter. For example, it is possible toimplement the modem at site of the assisted user using a digital signalprocessor, or DSP, integrated circuit. Such a DSP chip can be programmedto recognize to separate voice signals from text communications signalsand to pass along only the voice signals to the speaker at the locationof the assisted user. It is also contemplated that the telephone linkagebetween the relay and the assisted user could be a digital connectionrather than an analog telephone line. For example, the linkage betweenthe relay and the assisted user could be a digital wireless linkage oran internet protocol linkage, wired or wireless. In either of theseinstances, the appropriate protocol calls for the transmission ofpackets of digital data, the packets being marked to indicate the typeof information carried in the packets. The packets might contain, forexample, voice signals which have been digitized or might contain simpledigital information representing the text string of a conversation. Inthe instance of such a digital packet type communication protocol, thefiltering out of the digital text information could be implementedsimply by only converting the packets marked as voice back into sound.Packets marked as containing digital information would be used torecover the text information for display to the assisted user.

[0038] Captioned Telephone.

[0039] As mentioned earlier, captioned telephone is a technologyintended to provide text assistance to assisted user during the courseof a voice telephone communication session. As will become apparent fromthe discussion below, a captioned telephone connection can be a singleline connection or a multiple line connection. There are advantages anddisadvantages of each approach. A single line connection is known in theart, as exemplified by U.S. Pat. No. 6,075,842, mentioned previously.The advantages and details of the use of a multiple line connection aredescribed below. The discussion of captioned telephone will begin with asingle line connection.

[0040] Shown in FIG. 4 is an illustration of how a typical telephonecall involving a single line captioned telephone would be set up. Thehearing user at telephone 62 communicates through a telephone line 64with the relay, indicated at 66. The relay, a re-voicing relay,communicates through a telephone line 68 with the assisted user. At thesite of the assisted user is a telephone 70 used by the assisted userand also a captioned telephone device 72. The telephone 70 isconventional. The captioned telephone device 72 is constructed toaccomplish two objectives. One objective is to filter, or separate, thedigital signals carrying the text information from the voice signal. Theother objective is to take the digital signals and create a visualdisplay of the text information for the assisted user. This device isthus intended to assist the user to understand a greater portion of theconversation by providing a visually readable transcription of the textof the telephone conversation so that the assisted user can read anywords that he or she cannot hear properly. While the captioned telephonedevice 72 of FIG. 4 is illustrated as a separate stand-alone device fromthe telephone 70, those of skill in the art can readily appreciate thatthe two functions can be combined in a special capability telephonestation. Such a special purpose station can both provide conventionaltelephone service and also include a display which can be used toprovide captioning for an assisted user.

[0041] The two line captioned telephone is indicated in FIG. 5. In itssimplest embodiment, the two-line captioned telephone can beconceptually thought of as a personal interpreter used to provide textassistance to a telephone user. Thus in FIG. 5, the telephone of thehearing user 62 and the telephone of the assisted user 70 are connecteddirectly by a simple single telephone connection, indicated by thetelephone line 64. The assisted user then uses a personalinterpreter/captioned telephone (“PICT”) device 74 to connect to a relay76 over a second telephone line or connection, indicated at 78. Theremust be some form of communication between the telephone of the assisteduser 70 and the PICT device 74, so that the voice of the hearing usercan be transferred to the second telephone line 78 to the relay 76. Atits simplest, the connection can be a simple acoustic connection, suchas placing the sound pick-up microphone of the PICT device in such alocation that the sound of the hearing person's voice is picked up andtransmitted to the relay. A microphone connected to the PICT device 74could be placed on the handset of the assisted user's telephone 70.However, it is preferred that the PICT device be electronicallyconnected to the telephone of the assisted user so that the voice of thehearing user can be transmitted to the relay call assistant with minimalinterference or noise. The PICT device includes a visually readabledisplay and suitable decoding electronics and software so that thedevice can receive and display a text stream received from the relayover the second telephone line.

[0042] The main advantage of the two-line approach to captionedtelephone is that the captioned telephone service can be added to atelephone call already in progress. By contrast, to use single linecaptioned telephone for an incoming call, the call must directed throughthe relay to the assisted user at the beginning of the call. For atwo-line captioned telephone call, however, the call can be started as anormal telephone communication session, without the relay or thecaptioning. Then, if the assisted user decides captioning would behelpful to his or her understanding of the conversation, the captioningservice can be added while the call is in progress. To add thecaptioning service, the assisted user simply has the PICT device dialthe relay over the second telephone line. The voice of the hearing useris then transmitted over the second telephone line to the relay. Therelay converts the voice to text and the text stream created by therelay returns to the assisted user, also over the second telephone line.Note that in this arrangement, as indicated in FIG. 5, voice and textare transmitted in opposite directions over a single telephone linebetween the PICT device and the relay, voice going to the relay and textreturning on the same line. This arrangement of voice and text is thesame as used for the personal interpreter.

[0043] One device intended to implement the two-line captioned telephoneapproach is illustrated in FIG. 6. The PICT device 80 of FIG. 6 is ananalog device connected in series between the telephone line 64 to thehearing user and the telephone 70 of the assisted user. As seen in FIG.6, the telephone line 64 is connected directly through the PICT device80 to the telephone 70. However, that telephone connection is alsoconnected through a transformer 82 to a 2-wire to 4-wire converter 84.Connected to the output of the converter 84 is a low pass filter 86, theoutput of which is the input to an amplifier 88. The output of theamplifier 88 is connected to a 2-wire to 4-wire converter 89 whichconnects, in turn, to the telephone line 78 which connects to the relay(not shown here). The input signals from the telephone line 78 areconnected to a hi-pass filter 90, the output of which passes through areceive modem 92 to a microcontroller 94. The microcontroller operatesthe display 96. The microcontroller also includes the capability togenerate DTMF dialing tones and impress those tones on the telephoneline 78 when it is time to dial the relay. Thus here the single PICTdevice 80 connects to two telephone lines. A call can be initiated orreceived by the assisted user in the same manner as with a conventionaltelephone, using only the first telephone line 64. When the user wishesto invoke the captioning service, the assisted user presses a button,97,on the PICT device 80 that causes the device to automatically dial therelay on the second telephone line 78. The PICT device automaticallytransmits the voice signals on the first telephone line to the secondtelephone line (through the low-pass filter 86) for transmission to therelay. At the relay, the voice is converted into text and a text streamis sent back down the telephone connection 78 to the PICT device. ThePICT device filters the text data, at the hi-pass filter 90, andtransmits it to the microcontroller 94 for display to the user. None ofthe text data, and no digital carrier signals, are heard by either thehearing user or the assisted user. Normally the hearing user does noteven need to be aware that captioning is being used on the call.

[0044] This example assumes that the digital carrier signals for thetext message are carried on high frequencies, and the low pass and highpass filters are used to separate voice from text. While this filteringcan be implemented as an analog filter, in many telephonic systemstoday, digital forms of communication are used. Using a digitalcommunication protocol between the relay and the PICT device,communication is in the form of digital data packets of either text ordigitized voice. In that event, the logical filtering simply consists ofnot creating any acoustic noise from the packets designated as carryingtext.

[0045] Several physical versions of the PICT device are contemplated. Inthe version illustrated in FIG. 5, the PICT device is a stand-aloneappliance inserted in series between the telephone of the assisted userand the telephone jack on the wall of the user's home. It may also beconvenient for the user to package the PICT device as a single two-lineunit intended to provide both conventional telephone service, as well ascaptioned telephone services. The device could look like a normaltelephone, but have a display. The captioned telephone feature would beactivated by a button or key, which would cause the unit to dial apre-selected relay and set up a captioning session, whether or not acall on the other line is yet in progress. The voice signals incoming onthe first telephone line from the hearing user would be transferredelectronically to the telephone line to the relay. Text signals would betransmitted on that same telephone line back from the relay to the PICTdevice, which would display that text for the assisted user. Since there-voicing relay operates with only a very slight time delay, the textdisplay for the assisted user would trail the hearing person's voiceslightly, but the text would still assist the user in comprehending theconversation.

[0046] It is a desirable feature of the two-line captioned telephonearrangement that the use of the captioning service is transparent to thehearing user. The hearing user would dial to connect to the assisteduser as with any other telephone user. The assisted user invokes thecaptioned telephone service without the need to involve the hearing userat all. The service can be used equally well and transparently for bothincoming and outgoing calls.

[0047] It is also envisioned that the captioned telephone service can beimplemented in a way that uses three telephone lines. One line is forvoice communications with the hearing user. The second line is directlyconnected to the first line so that the voice of the hearing user istransmitted to the relay. The third line is a connection to transmit thetext stream from the relay to the station of the assisted user.

[0048] In the implementation of a relay providing captioned telephoneservice, the relay receives the voice of the hearing user and transmitsboth a digital text message stream and the voice of the hearing userover a telephone connection to the station of the assisted user. It isto be understood, however, that a conventional telephone single lineconnection is only one example of a telephonic connection that can beused in this arrangement. Digital wireless connection, or PCSconnection, or even internet protocol (IP), wired or wireless connectioncan be used to connect the relay to the assisted user, so long as theconnection in capable of trasmitting voice to that user.

[0049] Thus the term telephone line as used in this specification isintended not only to apply to a traditional land-line two-wire telephoneline, but also to all equivalents that offer similar functionality. Eachof the telephone lines could be, for example, a portion of the bandwidthof an ISDN or DSL service. The telephone line could be an analog ordigital cellular telephone link or a PCS connection. The PICT devicecould also be connected to the internet communication in IP, and in thatevent the two telephone lines would simply be simultaneous digital dataexchange with two remote locations.

[0050] It is to be understood that the present invention is not limitedto the particular illustrations and embodiments disclosed above, butembraces all such modified forms thereof as come within the scope of thefollowing claims.

I/we claim:
 1. A method for providing captioned telephone service to an assisted user communicating with a hearing user, the assisted user using a telephone station to communicate with a relay having speech to text translation capability, the method comprising the steps of initiating a telephone call over a first telephone line between the assisted user and the hearing user; the assisted user initiating a telephone connection over a second telephone line to the relay; transmitting the voice of the hearing user over the second telephone line to the relay so that the relay can convert the words of the hearing user into text; transmitting the text created by the relay back to the station of the assisted user over the second telephone line; and displaying the text to a captioned telephone display device within sight of the assisted person such that captioning of the communication session is provided to the assisted user.
 2. A method as claimed in claim 1 wherein the step of initiating the telephone call is performed by the assisted user.
 3. A method as claimed in claim 1 wherein the step of initiating the telephone call is performed by the hearing user.
 4. A method as claimed in claim 1 wherein the step of transmitting the voice of the hearing user is made through an electronic connection between the first telephone line and the second telephone line.
 5. A method for providing captioned telephone service to an assisted user communicating over a first telephone line with a hearing user using a captioned telephone device, the method comprising the steps of while the assisted user is conversing over the first telephone line with the hearing user, the assisted user operating a control on the captioned telephone device to initiate a captioning service; the captioned telephone device initiating a telephone connection over a second telephone line to the relay; the captioned telephone device transmitting the voice of the hearing user over the second telephone line to the relay so that the relay can convert the words of the hearing user into text; the relay transmitting the text created by the relay back to the station of the assisted user over the second telephone line to the captioned telephone device; and the captioned telephone device displaying the text within the sight of the assisted person such that captioning of the words spoken by the hearing user is provided to the assisted user.
 6. A method as claimed in claim 1 wherein the step of transmitting the voice of the hearing user is made through an electronic connection between the first telephone line and the second telephone line. 